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FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences,  through  a 
Memorandum  of  Agreement  with  the  National  Guard  Bureau  (NGB)  and  the  Army  National 
Guard  (ARNG),  provides  technical  support  to  NGB  initiative  on  distance  learning  technologies. 
One  form  of  this  support  is  the  evaluation  of  the  training  effectiveness  and  costs  of  alternate  forms 
of  instructional  delivery. 

The  present  report  demonstrates  that,  in  certain  cases,  low-cost  alternatives  to  classroom 
instruction,  namely  audio  teletraining,  can  achieve  remarkable  cost  savings  while  maintaining  a 
high  quality  of  instruction.  Although  there  are  many  military  tasks  for  which  audio  teletraining 
would  be  inappropriate,  such  as  any  task  with  a  hands-on  component,  it  could  serve  well  for 
routine,  knowledge-based  tasks  that  emphasize  print  media.  Such  is  the  case  for  the  tasks  trained 
to  unit  clerks  in  the  ARNG  through  the  Professional  Education  Center,  Camp  Robinson,  AR.  The 
present  report  provides  a  cost-effectiveness  analysis  of  audio  teletraining  for  the  Unit  Clerk 
Course.  Its  results  should  be  factored  in  to  considerations  of  the  mix  of  training  technologies 
appropriate  for  cost-effective  distance  learning  delivery. 


ZITAM.  SIMUTIS 
Technical  Director 


EDGAR  M.  JOHNSON 
Director 
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AUDIO  TELETRAINING  FOR  UNIT  CLERKS;  A  COST-EFFECTIVENESS  ANALYSIS 
EXECUTIVE  SUMMARY _ 


Research  Requirement: 

In  1995  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) 
signed  a  Memorandum  of  Agreement  with  the  National  Guard  Bureau  to  serve  as  the  program 
manager  for  the  academic  component  of  the  Army  National  Guard  (ARNG)  distance  learning 
network.  As  set  forth  in  that  agreement,  ARI  agreed  to  conduct  evaluations  of  various  distance 
learning  alternatives.  This  particular  research  examines  the  cost  effectiveness  of  using  audio 
training  originating  from  an  instructional  site  at  the  Professional  Education  Center,  Camp 
Robinson,  AR,  to  various  remote  sites  situated  at  ARNG  facilities  across  the  country. 


Procedure: 

The  participants  were  n=225  soldiers  from  ARNG  units  nationwide.  Of  these,  n=l  18 
participated  in  the  audio  teletraining  version  of  the  3-week  Unit  Clerk  Course  and  n=107 
participated  in  the  same  course  in  residence  at  Camp  Robinson,  AR.  Objective  performance  data 
were  collected  from  written  tests  on  16  of  the  47  tasks  taught.  Cost  data  were  computed  for 
material  distribution,  audio  bridge,  equipment,  travel,  and  test  monitors. 


Findings: 

Students  in  the  audio  teletraining  group  had  a  93%  Go  rate  (on  first  attempt)  for  the  16 
performance  tests,  which  was  significantly  higher  (by  statistical  measures)  than  the  85%  Go  rate 
for  the  residence  group.  In  comparing  costs,  the  audio  teletraining  group  had  lower  training 
costs,  on  average  $1,135  per  student.  This  was  due  primarily  to  an  avoidance  of  travel  costs  for 
the  audio  teletraining  group.  Projected  on  the  annual  training  load,  the  ARNG  can  save  $292,404 
per  year  through  the  use  of  audio  teletraining  for  the  Unit  Clerk  Course. 


Utilization  of  Findings: 

This  research  demonstrates  the  cost-effectiveness  of  audio  teletraining  for  selected 
courses.  The  ARNG  and  the  Deputy  Chief  of  Staff,  Training  for  the  Training  and  Doctrine 
Command  have  been  briefed  on  these  results.  Audio  teletraining  is  one  tool  available  for  distance 
learning.  Army  trainers  should  consider  the  merits  of  training  through  a  low-cost,  audio-only 
mode  when  the  tasks  and  materials  are  appropriate. 
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AUDIO  TELETRAINING  FOR  UNIT  CLERKS:  A  COST-EFFECTIVENESS 

ANALYSIS 


Introduction 

The  Army  National  Guard  (ARNG)  has  the  challenge  to  maintain  a  well-trained 
and  ready  force  during  a  period  of  declining  training  resources.  One  option  to  meet  this 
challenge  is  the  application  of  distance  learning  technologies  as  a  training  method.  Of 
course,  the  quality  of  training  must  remain  high  and  the  costs  must  be  acceptable  in  order 
for  this  option  to  be  feasible.  The  purpose  of  this  report  is  to  document  an  effort  by  the 
ARNG  to  apply  a  low  cost,  distance  learning  tool,  audio  teletraining,  to  train 
geographically  dispersed  soldiers.  Specifically,  an  analysis  of  the  training  effectiveness 
and  costs  between  groups  trained  either  in  residence  or  through  audio  teletraining  for  the 
Unit  Clerk  Course  will  be  presented.  This  distance  learning  example  should  be  useful  as 
the  ARNG  examines  cost  factors  for  future  training  requirements. 

Distance  learning  is  not  new  to  the  Army.  Indeed,  the  Training  and  Doctrine 
Command  (TRADOC)  is  advocating  widespread  use  of  distance  learning  in  the  near 
future  (TRADOC,  1996)  and  the  ARNG  has  responded  by  developing  a  plan  to  connect 
classrooms  in  54  States  and  Territories  as  one  distance  learning  network  (National  Guard 
Bureau,  1996).  In  the  education  and  training  literature,  distance  learning  has  various 
definitions.  It  is  generally  regarded  as  learning  that  occurs  in  a  different  place  from  the 
teacher  (Moore  and  Kearsley,  1996).  Educational  television  is  a  classic  example.  Radio 
training  serves  as  an  even  earlier  example  (Woelfel  and  Tyler,  1945).  The  advent  of 
satellite  and  terrestrial  networks  as  well  as  the  widespread  use  of  personal  computers  and 
the  Internet  has  catapulted  distance  learning  into  the  forefront  of  educational  change. 
Indeed,  universities,  state  and  federal  agencies,  trade  organizations,  and  industry  now 
offer  distance  learning  format  for  thouszinds  of  courses.  A  motivating  factor  for  this 
change  is  cost.  In  many  cases,  it  is  cheaper  to  have  the  students  remain  where  they  are 
and  “project”  the  teacher  to  distant  classrooms. 
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The  least  expensive  form  of  such  “projection”  of  instruction  is  an  audio  link 
between  sites.  Audio  teletraining,  the  focus  of  the  present  report,  is  simply  a  high-quality 
conference  call  that  allows  people  at  multiple  locations  to  hear  and  speak  to  one  another 
over  ordinary  telephone  lines.  It  is  audio  conferencing  with  the  goal  of  training.  In  the 
research  literature  on  distance  learning,  audio  teletraining  has  received  much  less 
attention  than  various  forms  of  video-based  training,  such  as  CD-ROM,  video 
conferencing,  or  emerging  multimedia  technologies.  The  reason  is  due,  in  part,  to  the 
obvious  lack  of  a  visual  medium  that  an  instructor  can  display,  mark,  manipulate,  and 
control  in  instructionally  useful  ways.  Other  advantages  to  video  are  that  it  allows 
students  to  “travel”  to  new  places  or  observe  demonstrations  that  are  otherwise  costly  or 
dangerous  to  demonstrate.  In  cases  where  training  depends  on  a  visual  presentation, 
particularly  one  that  is  animated  or  needs  to  be  controlled  by  the  instructor,  audio 
teletraining  alone  would  be  unsatisfactory.  Such  a  visual  requirement  is  not  always  the 
case. 


Another  reason  why  audio  teletraining  has  received  little  attention  is  that  there  has 
been  an  inclination  in  distance  learning  to  replicate  the  classroom  setting  of  face-to-face 
delivery,  regardless  of  content.  Here  students  observe  both  the  instructor  and  the  material 
being  presented  and,  in  the  case  of  two-way  video,  instruetors  can  observe  the  students 
and  what  they  might  present.  Some  researchers  have  observed  that  the  inclination  to  the 
face-to-face  video  format  derives  from  the  fact  that  traditional  instruction  is  the  only 
familiar  instructional  frame  of  reference  for  designers,  developers,  and  decision  makers 
on  these  matters.  Furthermore,  there  is  the  perception  that  students  prefer  the  seeurity  of 
having  their  instruetor  in  close  proximity,  in  a  real  or  virtual  sense  (Hardy  and  Olcott, 
1995).  These  issues  are  more  organizational  than  instructional.  Nevertheless,  audio 
teletraining  might  be  well  suited  for  instruction  in  which  static  visual  information  can  be 
provided  to  students  in  advance  and  for  a  course  in  which  the  instruetion  is  sequential 
with  little  branching.  Such  is  the  case  for  training  unit  clerks  in  the  ARNG. 

A  description  of  the  Unit  Clerk  Course  will  be  presented  along  with  the  data 
colleetion  methods,  the  analytieal  approach,  and  the  results  concerning  the  cost 
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effectiveness  of  audio  teletraining  for  the  Unit  Clerk  Course.  The  discussion  section  will 
include  an  overview  of  distance  learning,  a  review  of  the  research  literature  on  the 
effectiveness  of  audio  teletraining,  and  an  accounting  of  the  results  in  the  present  study. 


Method 


Unit  Clerk  Course 


In  an  ARNG  unit,  the  unit  clerk  is  responsible  for  recording,  updating,  and 
maintaining  personnel  records  of  unit  members,  from  company  level  through  higher 
echelons.  These  records  relate  to  a  broad  range  of  topics,  such  as  finance,  hfe  insurance, 
personnel  qualification  records,  statements  of  medical  examination,  unit  manning  reports, 
and  discharge  certification.  Altogether,  forty-seven  tasks  are  trained.  The  rank  of  a 
typical  student  is  sergeant  with  a  military  occupational  specialty  (MOS)  of  71L  Unit 
Administrative  Specialist,  although  soldiers  in  paygrade  E-4  through  E-7  and  from  other 
MOS  can  participate.  In  the  past  fiscal  year  (FY96),  280  ARNG  soldiers  were  trained  as 
unit  clerks. 

Until  recently,  training  for  the  unit  clerk  was  conducted  entirely  at  the 
Professional  Education  Center  (PEC)  located  in  Camp  Robinson,  in  North  Little  Rock, 
Arkansas.  The  three  week  course  required  that  soldiers  travel  to  Camp  Robinson,  be 
billeted  and  provided  meals.  A  distance  learning  initiative  at  PEC  afforded  the 
opportunity  to  convert  the  course  to  a  distance  learning  format. 

Participants 

Participants  were  n=225  soldiers  serving  as  trainees  from  ARNG  units 
nationwide.  Of  these,  n=107  participated  in  the  resident  classroom  version  of  the  Unit 
Clerk  Course,  and  n=l  18  participated  in  the  audio  teletraining  version.  Assignment  to 
groups  was  based  on  which  month  the  soldier  was  scheduled  for  training,  so  that  all 
soldiers  trained  in  November  1995,  December  1995,  and  May  1996  were  trained  in  the 
resident  classroom  whereas  soldiers  trained  in  March  1996  and  August  1996  were  trained 
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through  audio  teletraining.  (Note:  This  sample  does  not  represent  every  student  who  took 
the  Unit  Clerk  Course  in  FY96.) 

Program  of  Instruction  and  Instructors 

The  same  program  of  instruction,  student  workbooks,  job  aids,  and  tests  were 
used  by  both  groups.  The  same  four  instructors  taught  the  course  to  each  group.  Each 
student  workbook  and  job  aid  were  uniquely  color  coded,  which  allowed  for  easy 
reference  in  the  audio  teletraining  format,  e.g.,  “refer  to  your  green  job  aid.” 

Testing 

Written  performance  tests  were  administered  for  16  of  the  47  tasks  taught.  The 
16  tasks  that  were  tested  were: 

1.  Prepare  DA  Form  1379,  Completing  a  Training  Certificate,  Unit  Record  Of 
Reserve  Training 

2.  Compute/V erify  Pay  Entry  B  asic  Date 

3.  ComputeA^erify  Retirement  Year 

4.  Prepare  DA  Form  4836:  Oath  of  Extension  or  Reenlistment 

5.  Determine  qualification  for  Promotion  and  award  of  MOS  using  Promotion 
Eligibility  Checklist 

6.  Prepare  DD  Form  93:  Record  of  Emergency  Data 

7.  Prepare  DA  Form  2- 1 :  Personnel  Qualification  Record 

8.  Prepare  DA  Form  2-1 :  Compute/verify  Date  of  Rank 

9.  Prepare  NGB  Form  22  Report  of  Separation  and  Record  of  Service  55  Series 
Discharge  Certificate 

10.  Prepare  DD  Form  1883  Survivor  Benefit  Plan  Election  Certificate 

1 1 .  Coordinate  medical  care  for  injured/ill  soldier 

12.  Determine/  Prepare/  Review/  Process  Incapacitation  Pay 
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13.  Prepare  DA  Form  2173:  Statement  of  Medical  Examination  and  Duty  Status, 
Prepare/Review  Line  of  Duty  Investigation,  Sworn  Statement 

14.  Prepare  DA  Form  2823  (Sworn  Witness  Statement) 

15.  Review  and  screen  NCO-E valuation  Report  for  errors 

16.  Maintain  (MARKS)  Modern  Army  Record  Keeping  System 

Data  Collection 

Data  collection  occurred  after  training  was  completed.  Enrollment  records, 
diagnostic  data,  performance  test  scores  and  cost  data  were  obtained  for  all  students  from 
the  ARNG  Professional  Education  Center  for  students  who  recently  participated  in  either 
the  audio  teletraining  or  resident  instructional  format.  No  additional  tests  or 
questionnaires  specific  to  this  study  were  given  to  the  students.  Instructors  for  the  Unit 
Clerk  Course  were  interviewed  for  additional  information  on  preparation  of  materials, 
conduct  of  the  training,  and  insights  into  the  nature  of  interactions  in  the  distance  learning 
format. 


Results 

The  comparative  effectiveness  of  the  two  forms  of  training  will  first  be  presented, 
followed  by  a  description  of  relevant  cost  factors.  This  will  be  followed  by  an  overall 
cost-effectiveness  analysis  between  the  audio  teletraining  and  resident  modes  of 
instructional  delivery. 
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Training  Effectiveness 

The  analysis  of  training  effectiveness  must  first  take  into  account  any  differences 
between  groups.  Perhaps  the  largest  single  factor  that  could  potentially  bias  the  results 
was  prior  knowledge  about  the  course  content.  Since  some  of  the  soldiers  had  familiarity 
with  the  responsibilities  of  a  unit’s  clerical  tasks,  due  to  the  fact  that  many  already  had  a 
primary  military  occupational  specialty  in  the  clerical  field,  their  performance  on  the 
diagnostic  test  for  unit  clerks  was  a  key  in  comparing  prior  knowledge  between  groups. 
The  diagnostic  test  administered  to  all  soldiers  at  the  beginning  of  the  course  found  a 
9.2%  Go  rate  for  the  audio  teletraining  group  and  a  10.6%  Go  rate  for  the  residence 
classroom  group  (t=.87,  not  significant).  Thus,  with  respect  to  prior  knowledge,  there 
was  no  difference  between  groups.  Any  differences  between  the  treatment  groups  (audio 
teletraining  or  residence)  could  not  be  attributed  to  differences  between  members  of  the 
groups. 


In  order  to  determine  if  performance  on  the  tasks  measured  immediately  after 
training,  and  hence  training  effectiveness,  was  different  between  the  two  groups,  a 
multivariate  analysis  of  variance  (MANOVA)  was  performed  (Tabachnick  and  Fidell, 
1989).  The  performance  variables  for  those  soldiers  who  received  a  Go  on  the  diagnostic 
test  were  recoded  as  also  getting  a  Go  on  the  first  attempt  after  training.  This  was  done  in 
order  to  preserve  a  reasonable  sample  size.  Otherwise,  soldiers  who  received  a  Go  on  as 
little  as  one  diagnostic  (but  took  the  first  test  on  the  remaining  15  task  variables)  would 
be  dropped  entirely  from  the  analysis  due  to  missing  data^  The  assumption  here  is  that  if 
a  soldier  received  a  Go  on  a  diagnostic  test,  he  or  she  would  also  have  received  a  Go  on 
the  test  after  some  additional  training,  which  seems  reasonable.  Table  1  displays  the 
performance  on  the  diagnostic  test,  the  test  given  immediately  after  the  first  round  of 
training  and  the  graduation  rate. 


1  Because  the  pCTcentage  of  subjects  getting  a  Go  on  the  diagnostic  was  not  significantly  different  between 
the  two  groups  (9.2%  vs.  10.6%)  as  described  above,  we  did  not  anticipate  that  the  inclusion  of  these  few 
data  points  would  alter  the  effects  of  the  MANOVA.  Furthermore,  the  analytical  alternative  of  dropping  all 
of  the  data  for  these  subjects  would  have  meant  that  the  MANOVA  would  be  restricted  to  scores  of  only 
those  individuals  who  did  not  receive  Go  scores  on  any  of  the  diagnostic  measures,  which  did  not  seem 
representative  of  our  sample. 
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Table  1 

Performance  Data  between  Groups 


Group 


Audio  Teletraining  Residence 

(n=118)  (n=107) 

Diagnostic  Test  9.2%  Go  10.6%  Go 

First  test  after  training  93.3%  Go  85.4%  Go 

Graduation  Rate  100%  100% 


For  the  multivariate  analysis,  which  tests  whether  there  was  a  statistically 
significant  difference  between  groups  on  the  first  test  after  training,  the  sixteen  task 
variables  were  entered  as  the  dependent  variables  (measured  as  a  Go  or  No  Go  score,  1  or 
0),  and  group  membership  (Audio  Teletraining  or  Residence)  was  entered  as  the 
independent  variable.  As  MANOVA  combines  the  dependent  variables  into  a  linear 
combination,  which  represents  task  performance,  it  was  clear  that  there  was  a  statistically 
significant  difference  between  groups  on  task  performance  (F(15, 196)  =  4.80,  p<.001). 

There  was  superior  learning  performance  exhibited  by  the  audio  teletraining 
group,  as  measured  by  first  time  Go  rate.  However,  there  was  equivalence  in  the 
graduation  rate  between  groups,  since  all  students  ultimately  passed  the  course.  Thus, 
although  students  learned  more  quickly  in  the  distance  learning  format,  in  a  course  of 
fixed  length  (3  weeks)  where  there  was  ample  time  for  retraining  and  retesting,  there  was 
equal  effectiveness  between  groups.  This  analysis  supports  many  previous  reports  that  a 
distance  learning  group  performed  as  well  as,  if  not  better  than,  a  resident  control  group. 

Cost  Analysis 

In  comparing  costs  between  two  training  delivery  methods,  one  must  first  identify 
which  cost  elements  are  stable  and  which  elements  differ.  The  differences  must  then  be 
quantified,  added  and  compared.  In  the  present  analysis,  the  differences  were  computed 
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on  a  per  capita  basis  and  then  projected  on  an  annual  basis,  based  on  training  loads 
provided  by  the  Professional  Education  Center.  All  costs  reported  are  in  Fiscal  Year 
1996  dollars. 

Stable  cost  elements.  The  following  cost  elements  were  assumed  to  be  identical 
between  the  two  treatment  groups. 

1.  Salary  for  students.  The  average  student  in  the  unit  clerk  course  was  at  the  E-5 
paygrade.  Although  there  might  have  been  a  slight  difference  in  the  average  step  within 
that  paygrade  between  the  two  samples,  the  small  difference  was  considered  arbitrary. 

2.  Instructors.  Since  the  same  four  instructors  taught  each  course,  this  cost 
element  was  stable.  That  is,  the  cost  for  the  same  instructors  to  teach  either  a  three-week 
resident  course  or  three-week  distance  learning  course  was  identical. 

3.  Course  materials.  The  costs  for  preparation  and  production  of  books, 
worksheets,  job  aids,  and  tests  was  the  same  for  either  group. 

Differing  cost  elements.  The  following  cost  elements  varied  between  groups. 

1.  Distribution  of  materials.  The  was  a  $5.00  charge  per  student  for  overnight 
delivery  of  course  materials  in  the  audio  teletraining  group.  Those  in  the  residence  group 
received  the  materials  in  the  classroom  at  the  beginning  of  the  course. 

2.  Audio  bridge.  There  was  a  $.04  per  minute  per  site  charge  for  connection  to 
the  audio  bridge.  For  a  five  and  one-half  hour  instructional  day,  this  translates  to  $13.20 
per  day  per  site  ($.04  per  minute  multiplied  by  330  minutes).  For  15  days  of  instruction, 
this  multiplies  to  $198  per  site.  Based  on  recent  class  iterations,  there  were  ten  sites  per 
class  accommodating  an  average  of  59  students  per  class,  so  this  computes  to  an  average 
of  $33.56  per  student  per  class  ([$198  per  site  multiplied  by  10  sites]  divided  by  [59 
students]). 

3.  Travel.  For  students  in  the  resident  classroom  group,  the  average  travel  cost 
was:  airfare  $700,  lodging  $342  (18  days  at  $19  per  day),  and  meals  $92.87  (18.5  days  at 
$5.02  per  day),  for  a  total  of  $1 , 134.87.  For  students  in  the  audio  teletraining  group,  the 
average  travel  cost  for  the  three  weeks  of  training  was  $18.  (Note:  Most  of  these  students 
traveled  to  their  local  armories  or  were  within  reasonable  commuting  distance  of  an 
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armory  where  instruction  was  offered.  In  these  cases,  there  was  no  travel  charges.  A  few 
students  had  mileage  or  overnight  lodging  charges.  The  $18  average  for  a  class  of  59 
students  was  based  on  a  compilation  of  data  from  a  recent  iteration  in  which  one  student 
required  lodging  and  several  required  mileage.) 

4.  Equipment.  The  audio  teletraining  classroom  required  a  high  quality  audio 
conferencing  system,  which  cost  $500  per  site.  This  conferencing  system  can  have  many 
uses  beyond  audio  teletraining  for  the  armory  and  its  units.  Hence,  an  assumed  effective 
capital  expenditure  of  $250  per  site  wasmade,  allowing  for  50%  usage  for  other  purposes 
such  as  conference  calls  to  state  headquarters.  Also,  an  assumption  of  a  5-year  life  with 
no  salvage  cost  was  made.  A  final  assumption  was  that  each  site  would  conduct  three 
audio  teletraining  classes  per  year,  comparable  in  size  to  the  Unit  Clerk  Course.  These 
assumptions  translate  into  a  cost  of  $2.82  per  student  ([$250  per  site  divided  by  5  years] 
divided  by  [3  iterations  per  year  multiplied  by  5.9  students  per  iteration]). 

5.  Test  monitor.  Each  remote  site  required  the  duties  of  a  test  monitor  to  perform 
an  audio  check  prior  to  the  start  of  the  course,  take  roll  call  on  the  first  day  of  instruction, 
and  serve  as  a  test  control  officer  on  three  occasions  during  the  course.  The  latter 
function  requires  the  administration,  scoring  and  reporting  of  individual  student  scores  to 
the  origination  site  at  the  Professional  Education  Center.  Total  time  required  for  the  test 
monitor  function  was  14  hours.  Typically,  a  soldier  at  paygrade  E-7  served  as  the  test 
monitor,  although  paygrades  E-6  emd  E-8  have  also  been  employed.  The  total  time  for 
the  test  monitor  at  each  site  was  14  hours.  Additionally,  there  was  a  $5.00  cost  for 
providing  tests  and  answer  keys  to  each  test  monitor.  The  cost  for  this  function  was  $184 
per  site.  Computed  on  a  per  student  basis  (5.9  students  per  site)  this  cost  element  was 
$31.19. 
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Table  2 

Cost  elements  differine  between  srouns  on  oer  student  basis 

Audio  teletrainine 

Group 

Residence 

Distribution  of  materials 

+  $5.00 

Audio  bridge 

+  $33.56 

Travel 

+  $18.00 

+  $1,134.87 

Equipment 

+  $2.82 

Test  monitor 

+  31.19 

Total  $90.57  $1,134.87 

Difference  $1,044.30 


Cost  avoidance.  The  cost  element  data  are  presented  in  Table  2.  A  computation 
of  cost  avoidance  was  derived  on  a  per  capita  basis  on  the  cost  elements  identified  above. 
This  figure  was  then  multiplied  by  the  training  load  for  a  full  training  year.  For  students 
in  the  audio  teletraining  class,  there  were  unique  costs  of  $90.57  per  capita  (for 
distribution,  audio  bridging,  audio  conferencing  system,  and  mileage).  For  students  in 
residence,  there  were  unique  costs  of  $1,134.87  per  capita  (for  travel).  The  net  savings 
per  student,  then,  is  $1,044.30.  Multiplied  across  the  annual  training  load  of  280 
students,  the  overall  cost  avoidance  is  $292,404  per  year. 


Discussion 

In  a  recent  study  on  the  use  of  technology  for  training  and  education  in  the  Army, 
the  Army  Science  Board  recommended  that  the  Army  “continue  to  develop  and  acquire 
modern  classroom  technology,  but  emphasize  a  move  toward  distance  learning”  (Grum, 
et  al.,  1995,  p.37).  The  results  of  the  present  study  lend  further  support  to  this  position: 
the  Army  National  Guard  Unit  Clerk  Course  can  be  taught  as  effectively  through  distance 
learning  at  a  substantially  lower  cost.  After  a  general  review  of  distance  learning  in  the 
military,  the  discussion  will  explore  the  findings  of  the  present  study  in  more  detail  and 
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offer  recommendations  of  when  to  use  audio  teletraining  as  a  training  tool. 

Distance  Learning 

U.S.  Military  training  has  employed  distance  learning  in  various  forms.  For 
example,  the  Chief  of  Naval  Education  and  Training  (CNET)  now  has  videoteletraining 
(VTT)  in  operational  use  through  the  CNET  Electronic  Schoolhouse  Network.  This  VTT 
system  uses  an  interactive  two-way  video  and  audio  television  system  that  allows 
students  at  remote  sites  to  participate  in  training  from  an  origination  site  where  other 
students  are  located  with  the  instructor.  Research  on  the  effectiveness  of  the  CNET 
Electronic  Schoolhouse  Network  has  demonstrated  equal  effectiveness  between  resident 
classroom  and  remote  site  (Simpson,  Wetzel  &  Pugh,  1995,  Wetzel,  1995,  Wetzel, 
Radtke,  Parchman  &  Seymour,  1996).  The  Air  Force  has  a  similar  VTT  system,  and 
NATO  countries  have  also  been  using  distance  learning  in  various  forms  successfully 
(Seidel  and  Chatelier,  1994).  An  evaluation  of  service  plans  for  using  distance  learning 
technologies  to  train  Reserve  Component  personnel  reported  that  distanee  learning  has 
the  potential  to  provide  effective  training  and  reduce  costs  (Metzko,  Redding  &  Fletcher, 
1996). 


Studies  of  distance  learning  in  the  Army  have  demonstrated  similar  results.  For 
example,  the  application  of  the  distance  learning  tool  of  asynchronous  computer 
confereneing  for  the  Engineering  Officer  Advance  Course  showed  no  difference  between 
resident  and  distance  learning  students  on  objective  measures  of  performance,  but  the 
distance  learning  course  cost  less  than  the  resident  version  (Phelps,  Wells,  Ashworth  & 
Hahn,  1991).  In  an  example  with  soldiers  from  the  Reserve  Component  (RC),  the 
application  of  two-way  interactive  video  for  training  three  military  occupational 
specialties  to  RC  soldiers  demonstrated  performance  as  high  as  soldiers  trained  in  a 
resident  mode  (Bramble  and  Martin,  1995).  In  another  study  that  examined  the  training 
effectiveness  of  one-way  and  two-way  video,  Lehman  and  Kinney  (1992)  report  that  one- 
way  VTT  performed  better  than  two-way  VTT  for  a  course  on  common  leader  training  to 
noncommissioned  officers  from  the  RC.  In  support  of  distance  learning,  the  Army  has 
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maintained  distance  learning  facilities  for  several  years,  featuring  a  two-way  interactive 
system  known  as  the  Teletraining  Network,  or  TNET  (Schall,  1991). 

On  a  broader  level,  distance  learning  is  viewed  by  the  Army  as  the  “delivery  of 
standardized  individual,  collective,  and  self-development  training  to  soldiers  and  units  at 
the  right  place  and  right  time  through  the  application  of  multiple  means  and  technology” 
(TRADOC,  1996).  Because  of  the  numerous  training  technologies  this  definition 
implies,  it  is  useful  to  regard  distance  learning  from  a  toolbox  point  of  view.  That  is, 
there  are  different  instructional  tools  that  can  be  selected  and  applied  to  satisfy  a  training 
requirement.  The  choice  of  tool  depends  on  the  type  of  task  being  taught,  the  purpose  of 
the  training,  the  size  of  the  training  audience,  the  cost,  and  the  availability  of  facilities, 
equipment,  and  courseware.  Such  a  toolbox  allows  for  multiple  media  to  be  used  in 
appropriate  combinations  for  a  course.  The  variety  of  tools  available  for  the  ARNG 
distance  learning  network  permits  considerations  of  training  effectiveness  and  costs  as 
factors  in  deciding  on  which  media  functions  to  use. 

Instructional  Issues 


One  interesting  facet  of  audio  teletraining  is  the  autonomy  that  balances  across  the 
instructional  sites.  Each  student  is  known  to  the  instructors  solely  on  the  basis  of  test 
performance  and  how  well  they  respond  to  spoken  questions.  Since  students  cannot 
know  whether  they  will  be  the  next  to  be  called  upon  to  answer  a  question,  there  is  a 
perception  on  the  part  of  the  instructors  that  students  in  the  audio  teletraining  mode  attend 
to  the  instruction  more  carefully  than  their  resident  mode  counterparts.  Furthermore,  the 
accuracy  of  responses  reflects  on  both  the  student  and  his  remote-site  “peer  group,”  each 
of  whom  is  a  member  of  that  state’s  Army  National  Guard.  A  cohesion  between  students 
at  a  remote  site  might  form  that  is  not  unlike  the  cohesion  that  forms  within  platoons  of  a 
line  company. 

Research  on  cohesion  within  small  units  of  the  Army  has  demonstrated  that  the 
higher  the  cohesion,  the  higher  the  motivation  to  perform  well  (Siebold  and  Kelly,  1988). 
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If  it  is  the  case  that  the  cohesion,  and  thus  motivation,  of  the  students  at  the  remote  site 
was  greater  than  that  of  the  resident  students,  the  higher  performance  on  first  test  after 
training  (93%  versus  85%)  could  be  attributed  to  a  higher  level  of  motivation  at  the 
remote  sites.  Since  students  in  the  residence  course  are  not  segregated  into  small  groups, 
the  condition  for  small  unit  cohesion  to  form  is  not  as  likely  to  occur. 

Another  instructional  consideration  in  the  present  study  concerns  the  use  of 
different  learning  strategies.  Previous  research  has  demonstrated  that  students  are  not 
conscious  of  the  newly  adapted  learning  strategies  they  use  to  acquire  skills  and 
knowledge  in  the  unconventional,  distance  learning  classrooms  (Eastmond,  1995).  The 
need  to  stay  in  close  step  to  the  instruction,  since  any  student  may  be  called  upon  next, 
may  have  led  to  development  of  an  attentive  learning  strategy  that  improved  knowledge 
acquisition  as  reflected  in  the  higher  first  time  Go  rates  in  Table  1. 

Cost  Effectiveness 


The  cost  effectiveness  of  technology-based  training  in  the  military  has  been 
previously  documented  (Orlansky  and  String,  1979,  Wisher  and  O’Hara,  1981,  Fletcher, 
1990).  Generally,  technology-based  training  leads  to  improved  skill  and  knowledge 
acquisition  rates.  In  the  present  study,  although  there  was  no  difference  in  the  overall, 
final  pass  rate  between  the  two  groups,  the  audio  teletraining  group  performed  better  than 
the  residence  group  on  the  performance  test  administered  immediately  after  training. 

This  finding  is  consistent  with  many  previous  findings  on  the  effectiveness  of 
technology-based  training.  Of  greater  importance  here,  however,  is  the  fact  that  the  audio 
teletraining  format  costs  much  less  than  the  residence  format.  Based  on  the  cost  and 
performance  factors  analyzed  in  the  present  report,  the  audio  teletraining  format  is  clearly 
cost  effective. 

Other  cost  advantages.  Not  quantified  in  the  results  section  are  other  benefits  that 
can  accrue  from  a  distance  learning  format.  The  first  is  in  productivity  during  training. 
The  course  was  five  and  one-half  hours  per  day.  For  students  in  the  distance  learning 


13 


format,  this  would  allow  another  two  and  one-half  hours  of  productive  work  time  for  the 
student  at  home  unit.  Of  course,  for  those  who  had  a  short  commute  or  used  the  time  to 
study  this  would  not  necessarily  be  the  case,  but  it  is  a  notable  advantage.  A  second 
benefit  is  avoiding  travel  to  training  sites,  which,  in  addition  to  saving  money,  avoids  the 
need  for  family  support  and  other  away-firom-home  considerations  for  the  individual 
soldier.  With  reserve  component  soldiers  being  required  increasingly  to  deploy  on 
missions  such  as  Operation  Joint  Endeavor,  Operation  Restore  Hope,  and  the 
Multinational  Force  and  Observers  in  the  Sinai,  less  time  away  from  home  for  training 
can  reduce  the  overall  time  away  from  family  and  community. 

Conclusions  and  Recommendations 

Audio  is  a  critical  component  of  nearly  all  distance  learning  technologies.  In  the 
face-to-face  instruction  that  occurs  through  videoteletraining,  for  instance,  low  ratings  of 
the  quality  of  the  audio  can  quickly  overshadow  the  quality  of  an  otherwise  excellent 
video  production^.  Ineffective  instruction  can  result.  The  importance  of  high  quality 
audio  for  facilitating  distance  learning  instruction  has  been  identified  repeatedly  in  the 
literature  (Garrison,  1990,  Hardy  and  Olcott,  1995).  Modem  audio  conferencing 
technologies,  procured  at  a  low  cost  and  capable  of  transmitting  over  highly  reliable 
telephone  lines,  offers  quality  audio  for  instructional  purposes.  Its  cost  effectiveness  has 
been  demonstrated  in  the  present  study  and  in  other  training  and  education  applications 
(Burge  and  Howard,  1990). 

The  use  of  audio  teletraining  to  deliver  the  Unit  Clerk  Course  is  a  cost  effective 
means  to  provide  this  training  to  soldiers  on  a  nationwide  basis.  The  estimated  cost 
savings  on  an  annual  basis  is  about  $292,000.  The  audio  teletraining  course  should  be 
scheduled  so  that  as  many  unit’s  as  is  feasible  have  the  opportunity  to  participate  in  this 
training. 

The  audio  teletraining  format  has  potential  for  cost  savings  for  other  training 
applications.  Courses  that  are  candidates  for  conversion  to  audio  teletraining  are  those 

^  In  an  analysis  of  single-day  training  events  conducted  for  the  ARNG  over  the  past  year,  quality  of  audio 
was  rated  lower  than  quality  of  video  and  there  was  a  statistically  significant  correlation  between  quality  of 
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that  involve  print  based  media  that  could  be  sent  in  advance  to  students.  The  candidate 
courses  should  also  be  highly  structured,  knowledge-based  (rather  than  hands-on  skills) 
and  have  periodic  paper-and-pencil  tests.  To  realize  a  considerable  cost  savings,  the 
course  should  eliminate  the  need  to  travel  from  an  armory  to  a  centralized  training  site. 

Finally,  the  fact  that  distance  learning  has  again  demonstrated  equal  or  superior 
instructional  effectiveness  to  traditional  classroom  training  should  no  longer  be 
surprising,  as  its  effectiveness  has  been  demonstrated  hundreds  of  times  (Russell,  1996). 
Other  factors  that  are  inherent  to  a  distance  learning  format,  such  as  a  higher  level  of 
motivation  and  the  adoption  of  a  new  learning  strategy  by  individual  students,  could  also 
have  contributed  to  the  superior  learning  performance  of  the  students  in  the  audio 
teletraining  group.  The  strongest  finding  in  the  present  study,  however,  is  the  substantial 
cost  savings  that  resulted  from  a  perfect  fit  between  a  routine  course  and  low-cost 
technology. 

There  is  a  continuing  need  to  research  and  evaluate  the  most  cost-effective  means 
to  deliver  distance  learning.  From  the  present  study,  the  “low-end”  technology  of  print 
and  audio  teletraining  was  well  suited  to  the  training  requirement  for  unit  clerks.  In  view 
of  the  range  of  training  tools  available  through  distance  learning  -  print,  videotape, 
computer-based  training,  CD-ROM,  video  teletraining,  audio  teletraining,  and  the 
Internet  -  there  needs  to  be  an  ongoing  effort  to  understand  the  best  match  between  task, 
training  tool  and  cost. 


audio  and  training  effectiveness  (Wisher,  Curnow  &  Gividen,  1997 ) . 
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